Solved: PySpark: Writing Parquet Files to the Azure Blob S...

您所在的位置:网站首页 databricks read parquet file from s3png Solved: PySpark: Writing Parquet Files to the Azure Blob S...

Solved: PySpark: Writing Parquet Files to the Azure Blob S...

2023-07-28 17:28| 来源: 网络整理| 查看: 265

Currently I am having some issues with the writing of the parquet file in the Storage Container. I do have the codes running but whenever the dataframe writer puts the parquet to the blob storage instead of the parquet file type, it is created as a folder type with many files content to it.

One note to it, I tried searching various ways in the internets that it seems this is default creation when using pyspark and I can see in the folder created there was a file parquet with a snappy add into it (refer to the screenshots below)

 If this is a default creation to pyspark code, how can I write a single parquet format that will do some splitting or creation of folder? any recommendations? on how to do it?



【本文地址】


今日新闻


推荐新闻


CopyRight 2018-2019 办公设备维修网 版权所有 豫ICP备15022753号-3